Skip to content

fix(memory): efficiency and correctness overhaul for extended + legacy memory - #88

Merged
jkyberneees merged 1 commit into
mainfrom
fix/memory-efficiency-and-smartness
Jul 22, 2026
Merged

fix(memory): efficiency and correctness overhaul for extended + legacy memory#88
jkyberneees merged 1 commit into
mainfrom
fix/memory-efficiency-and-smartness

Conversation

@jkyberneees

Copy link
Copy Markdown
Contributor

Consolidated fix for 21 bugs found in a memory-system audit (5 parallel audit agents, top findings re-verified against the code before fixing). All fixes include regression tests.

Correctness

Extended memory

  • Cap bypass on the scan-reject path (regression from fix(memory+guard): quarantine scan-rejected atoms and fix socket_path transport #87): scan-rejected atoms were quarantined before enforceCap ran, so a rejection storm could grow quarantine past max_size_mb and wedge the store (the evictor only evicts trusted atoms). Cap is now enforced on both quarantine paths.
  • ReturnAfterBreak summarized the 5 OLDEST atoms — backwards iteration over a newest-first list.
  • Final recall ordering discarded relevance: the union of literal + predicted results was re-sorted by pure RetentionScore (degenerating to sort-by-recency with extractor confidence defaulting to 1.0), throwing away the blended 0.6·sim + 0.4·retention score and the paid LLM rerank order. The union is now deduped by ID and sorted by composite score (docs/EXTENDED_MEMORY.md updated accordingly).
  • No dedup of extracted atoms — re-stated facts accumulated as duplicates. Adds now dedup by normalized text (refresh CreatedAt, keep higher confidence and original ID).
  • Eviction now removes association links; ForgetAtom works for quarantine-only atoms; atom truncation is rune-safe; quarantine TTL eviction no longer writes under a read lock; background user-model inference can't overlap itself or append duplicate pending reviews.
  • Fenced/preamble LLM JSON (```json wrappers) made extraction/inference/prediction silently produce nothing — a shared extractJSON helper now strips fences and extracts the first balanced JSON span.

Agent loop

  • Skill loader and episode recall re-ran on every loop iteration (up to 300×/turn) whenever the result was empty — the dedup key was only assigned on non-empty results.
  • Per-message dedup keys were never reset between Run/RunWithMessages — a REPL user sending the same text twice got no memory hooks the second time.

Legacy memory

  • merge_on_write dropped facts when two entries shared a 30-char prefix (substring Replace → ambiguous) — new index-based FactStore.ReplaceAt.
  • Episode Search query cache was never invalidated on write/promote/prune — promoted episodes stayed invisible to repeat searches.
  • BuildSystemPrompt skipped the extended user-state block when legacy facts were empty, and re-read fact files every iteration because the empty result was never cached.
  • The LLM ranker's explicit "none relevant" was overridden with unranked candidates; Consolidate bypassed the facts char cap (and persisted untrimmed entries).

Efficiency

  • Index rebuild storm: every per-turn atom add triggered a full vector-index rebuild (dirty flag per add) — now one rebuild per turn via a batch add path (single markDirty, single assoc.Persist).
  • Cold embedding cache per rebuild: index/episode rebuilds constructed a fresh embedder each time, so the HTTP embedder's cache never warmed and the entire corpus was re-embedded over the network per atom. The index now reuses one embedder; episode dedup + index share one via new embedding.Shared (HTTP backend only — RP is corpus-fitted and stays per-corpus).
  • Recall fan-out: one recall cost ~8 LLM calls (duplicate searches + reranks per predicted intent under a shared 5s budget) — now 2 (rerank only the literal query; type filtering over the existing candidate set).
  • Per-candidate file re-parsing in recall — one store load per query now.
  • Per-turn extraction blocked the ReAct loop (anaphora + 30s LLM call inline before the next think) — now async via MemoryManager.RunBackground (context.WithoutCancel), keeping the loop responsive.
  • Episode extraction died at CLI exit (fire-and-forget goroutine before os.Exit) — session-end work is now tracked and drained with a bounded 15s wait in Agent.Close, so episodes survive odek run/continue/REPL exit.

API additions (no signature changes)

MemoryManager.RunBackground, MemoryManager.WaitForBackground, FactStore.ReplaceAt, embedding.Shared. NewLLMRanker's "none" case returns an unexported sentinel error.

Verification

  • 30+ new regression tests (batch rebuild count, embedder reuse, composite ordering, cap enforcement on reject path, dedup keys, async extraction, CLI-exit drain, merge prefix collision, cache invalidation, fenced JSON, …)
  • go test ./... — 27 packages, all green
  • go test -race ./internal/memory/... ./internal/loop/... ./internal/embedding/... — green
  • golangci-lint run ./... — 0 issues

…y memory

Consolidates fixes for 21 audited bugs in the memory system.

Correctness:
- extended: scan-rejected atoms now go through enforceCap before
  quarantine (regression from #87: quarantine could exceed max_size_mb
  and wedge the store, since the evictor only evicts trusted atoms)
- extended: ReturnAfterBreak summarized the 5 OLDEST atoms (backwards
  iteration over a newest-first list) instead of the most recent
- extended: final recall union was re-sorted by pure RetentionScore,
  discarding the blended 0.6*similarity+0.4*retention score and the LLM
  rerank order; the union is now deduped by ID and sorted by composite
- extended: extracted atoms dedup by normalized text (refresh CreatedAt,
  keep higher confidence) instead of accumulating duplicates
- extended: eviction now removes association links (no dangling IDs),
  ForgetAtom works for quarantine-only atoms, UTF-8-safe truncation,
  quarantine TTL eviction no longer writes under a read lock
- loop: skill/episode recall no longer re-runs on every iteration when
  the result is empty (dedup key was only set on non-empty results)
- loop: per-message dedup keys reset between Run/RunWithMessages calls
  (a repeated identical REPL message previously skipped all memory hooks)
- legacy: merge_on_write no longer drops facts on 30-char prefix
  collisions (new index-based FactStore.ReplaceAt)
- legacy: episode Search query cache invalidated on write/promote/prune
- legacy: BuildSystemPrompt includes the extended user-state block even
  when legacy facts are empty, and caches the empty result
- legacy: LLM ranker's explicit 'none relevant' is honored instead of
  being overridden with unranked candidates
- legacy: Consolidate enforces the facts char cap and actually trims
- extended: fenced/preamble-wrapped LLM JSON (extractor, user-model
  inference, predictor) now parses instead of silently dropping output

Efficiency:
- extended: batch per-turn atom adds - one index rebuild per turn
  instead of one per atom, one assoc.Persist per batch
- extended: vector index reuses one embedder across rebuilds so the
  HTTP embedding cache actually warms (was a full-corpus re-embed over
  the network per added atom); same sharing for episode dedup + index
  via embedding.Shared (HTTP backend only; RP stays per-corpus)
- extended: recall fan-out cut from ~8 LLM calls to 2 (type filtering
  over the existing candidate set, no rerank for predicted intents)
- extended: one store load per recall query instead of a full atoms.json
  re-parse per candidate; AnaphoraResolve drops its duplicate search
- extended: per-turn extraction + anaphora run async via
  MemoryManager.RunBackground instead of blocking the ReAct loop
- legacy: session-end episode extraction/consolidation are tracked and
  drained with a bounded 15s wait in Agent.Close, so episodes survive
  CLI exit instead of dying with the process
- extended: background user-model inference guarded against overlap and
  pending-review duplicates

Tests: 30+ new tests across memory/extended/loop/embedding/odek; full
suite, -race on memory+loop+embedding, and repo-wide golangci-lint all
clean.
@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
odek 1931bfe Commit Preview URL

Branch Preview URL
Jul 22 2026, 09:49 AM

@jkyberneees
jkyberneees merged commit 261a4df into main Jul 22, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant